Evaluating Syntactic Annotation of Ancient Languages

Abstract In this paper we introduce an extended version of the Vedic Treebank ( vtb , Hellwig et al. 2020) which comes along with revisited and annotation guidelines. order to assess quality our annotations as well usability limits guidelines performed inter-annotator agreement test. The results show that between annotators is hampered by various factors, most prominently insufficient understanding content because cultural temporal gap incomplete knowledge grammar. An in-depth discussion disagreeing demonstrates setup workflow, too, has a major influence on agreement. We suggest some measures can help increase transparency consistency according current language when annotating Sanskrit, or ancient varieties in general.